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ABSTRACT 



This study sought to determine the extent to which 
instructional processes characterized by cognitive complexity influence the 
development of general cognitive abilities during the first, second, and 
third years of college. The sample for the study consisted of incoming 
first-year students at 18 four-year and 5 two-year colleges and universities 
located in 16 states chosen to represent differences in colleges and 
universities nationwide on a variety of characteristics, including 
institution type and control, size, location, student residence patterns, and 
student ethnic distribution. Initial data was collected in fall 1992 from 
3,840 students at 23 institutions; three follow-up data collections were done 
in spring 1993, 1994, and 1995. Controlling for factors such as precollege 
cognitive ability and academic motivation, ethnicity, gender, socioeconomic 
status, age, college experiences, work responsibilities, and pattern of 
courses taken, the extent to which students reported instructor use of 
higher-order questions on examinations, assignments, and in classroom 
discussions was significantly and positively associated with end-of-year 
cognitive development for each year of the study. Additional analyses 
suggested that the net cognitive impact of the instructional processes were 
general rather than conditional. Four data tables are appended. (Contains 25 
references.) (CH) 
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Abstract 



This study sought to detemiine the extent to which instructional processes 
charactenzed by cognitive complexity influences the development of general cognitive 
abilities during the first, second, and third year of college. Controlling for such factors as 
precollege cognitive ability and academic motivation, ethnicity, gender, socio-economic 
status, age, college experiences, work responsibilities, and the pattern of courses taken, 
the extent to which students reporting instructor use of higher-order questions on 
examinations, assignments and in classroom discussions was significantly and positively 
associated with end-of-year cognitive development for each year of the study. Additional 

analyses suggested that the net cognitive impact of the instructional processes were 
general rather than conditional. 
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The Influence of Instructional Processes on 



Student Cognitive Development 

Introduction 

A central mission of American higher. education has long been the development of 
students’ intellectual skills and abilities. This has become especially relevant in today’s rapidly 
changing society where content knowledge is quickly becoming obsolete, and xmdergraduate 
students are expected upon graduation to demonstrate their abilities to reason, evaluate, make 
decisions, and solve problems (McKeachie, 1986). The significance of this mission for higher 
education has been underscored by several national reports such as Involvement in Learning 
(National Institute of Education, 1984) 21 ^ 6 . Integrity in the College Classroom (Association of 
American Colleges, 1985) both of which have identified the development of critical thinking 
skills as a desired and necessary outcome of undergraduate education. More recently, in a report 
by one of the nations’ leading education-policy organizations. The Carnegie Foundation for the 
Advancement of Teaching, higher education is exhorted to reassess its approach to 
undergraduate education so that “the skills of analysis, evaluation, and synthesis will become the 
hallmarks of a good education, just as absorption of a body of knowledge once was.” 

There is a considerable body of evidence indicating that exposure to college in general has 
a significant impact on students’ cognitive development (Astin, 1993; McMillan, 1987, 
Pascarella and Terenzini, 1991; Pace, 1984). There is also evidence to suggest that specific 
aspects of the college experience contribute to students’ intellectual growth (e.g., curriculum 
design (Dressel and Mayhew, 1954), disciplinary majors and courses taken (Astin, 1993, Tsui, 
1999), students’ out of class experiences with peers (Edison, 1997; Terenzini et al. 1996), 
students’ college residence (Pascarella et al., 1993), the quality of student effort (Pace, 1984), 



students’ out of class interactions with faculty (Terenzini et al., 1995), student involvement in 
active learning (Astin, 1993), students’ social and academic integration (Terenzini and Wright, 
1987; Li et al., 1999), and institutional context (Hagedom et al., 1999). 

Within the literature on teaching in higher education, there is a body of research that 
addresses the relationship between instructional practices and students’ growth and development 
(Dunkin and Barnes, 1986). In a meta-analysis of twenty-seven studies examining the influence 
of specific instructional or curricular approaches on student cogmtive development, McKeachie 
et al. (1986) concluded that certain elements of instruction can, in fact, foster critical thinking 
skills. Specifically, they found that students’ cogmtive development can be enhanced through 
instructional processes such as those that stress student discussion, emphasize problem solving 
tasks, or actively engage students in the use of higher-order thinking. Similarly, in a 
cornprehensive review of the relevant literature, Pascarella and Terenzini (1991) found that, 
while no one instructional practice appears to consistently enhance students’ cogmtive 
development, instructional practices that' involve students and faculty in the learning process at a 
reasonably high intellectual level may enhance critical thinking in students. In a more recent 
review of the literature on critical thinking, Tsui (1998) reports somewhat similar findings 
regarding the impact of certain types of instructional practices on student self-reported gains in 
critical thinking. General findings from this review indicate that students make greater gains in 
critical thinking in courses where instructional practices engage students in problem solving, 
higher order thinking and inquiry, and the use of reading and writing to create meamng. What is 
clear from these reviews is that the cognitive level of classroom instruction appears to enhance 
students’ development of higher order thinking. Efforts to identify the cognitive level of 
classroom instruction have relied mainly on Bloom’s (1956) Taxonomy of Educational 
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Objectives which categorizes the intended learning outcomes in a hierarchy according to the 
intellectual skills and abilities they elicit from students. In ascending order, the six levels of 
cognitive complexity are: knowledge, comprehension, application, analysis, synthesis, and 
evaluation. The underlying principal of this taxonomy is that instructional practices that engage 
students in thinking at higher cognitive levels can enhance students’ ability to think critically. 

Employing data from the National Study of Student Learning, the present study sought to 
add to our knowledge of the factors that influence students’ general cogmtive development. The 
study had two purposes. First, it attempted to determine the net effects of teaching practices that 
elicit higher-order thinking on students’ self-reported gains in general cogmtive skills and 
abilities. Drawing from Bloom’s taxonomy (1956), higher -order cognitive instruction is that 
which requires students to manipulate and synthesize information, while lower-order cognitive 
instruction elicits from students verbatim recall or recognition of factual information (Brown and 
Atkins, 1988). Second, the study attempted to determine the extent to which the effects of the 
variable representing the cognitive level of classroom instruction differed in magnitude for 
students with different background characteristics. 

Research Methods 

Samples 

Institutional sample . The sample in this study consisted of incoming first-year students at 
18 four-year and 5 two-year colleges and universities located in 16 states. Institutions were 
chosen from the National Center on Education Statistics Integrated Postsecondary Education 
Data System (IPEDS) data to represent differences in colleges and umversities nationwide on a 
variety of characteristics, including institutional type and control (e.g., private and public 



research universities, private liberal arts colleges, public and private comprehensive universities, 
two-year colleges, historically Black colleges), size, location, patterns of student residence, and 
the ethnic distribution of the undergraduate student body. Our sampling technique produced a 
sample of institutions with a wide range of selectivity. For example, we included some of the 
most selective institutions in the country, and some that were essentially open admission. 

Student sample . The individuals in the sample were students participating in the National 
Study of Student Learning (NSSL), a large longitudinal investigation of the factors that influence 
learning and cognitive development in college. The initial sample was selected randomly from 
the incoming first-year class at each participating institution. The students in the. sample were 
informed that they would be participating in a national longitudinal study of student learning and 
that they would receive a cash stipend for their participation in each data collection. They were 
also informed that any information they provided would be kept confidential and never become 
part of their institutional records. 

Data Collection 

Initial data collection . The initial data collection was conducted in the Fall of 1992 with 
3,840 students from the 23 institutions. The data collected included an NSSL precollege survey 
that sought information on student demographic characteristics and background, as well as 
aspirations, expectations of college, and orientations toward learning. Participants also 
completed Form 88A of the Collegiate Assessment of Academic Proficiency (CAAP), developed 
by the American College Testing Program (ACT) to assess selected general skills typically 
acquired by students during the first two years of college (ACT, 1989, 1991). The total CAAP 
consists of five 40-minute, multiple-choice test modules: reading comprehension, mathematics, 
critical thinking, writing skills, and science reasoning. The reading comprehension. 




7 



■ 4 



mathematics, and critical thinking modules of the CAAP were administered during the Fall 1992 
data collection. A brief description of each follows. 

The CAAP reading comprehension test is composed of 36 items that assess reading 
comprehension as a product of skill in inferring, reasoning, and generalizing. The test consists of 
four 900-word prose passages designed to represent the level and kinds of reading students 
commonly encounter in college curricula, including topics in fiction, humanities, social sciences, 
and natural sciences. The KR-20 internal consistency reliability for the reading comprehension 
test ranges between .84 and .86 (ACT, 1989). 

The mathematics test consists of 35 items designed to measure a student’s ability to solve 
mathematical problems. The test emphasizes quantitative reasoning, rather than formula 
memorization, and includes algebra (four levels), coordinate geometry, trigonometry, and 
introductory calculus. The KR-20 reliability coefficients for the mathematics test range between 
.79 and .81 (ACT, 1989). 

The critical thinking test is a 32-item instrument designed to measure a student’s ability 
to clarify, analyze, evaluate, and extend arguments. The test consists of four passages in a 
variety of formats (e.g., case studies, debates, dialogues, experimental results, statistical 
arguments, editorials). Each passage contains a series of arguments that support a general 
conclusion and a set of multiple-choice test items. The KR-20 reliability coefficients for the 
critical thinking test range between .81 to .82 (ACT, 1989). In a pilot test of instruments for use 
in the NSSL, the critical thinking test of the CAAP correlated .75 with the total score on the 
Watson-Glaser Critical Thinking Appraisal (Pascarella, Bohr, Nora, & Terenzini, 1995). 

First follow-up data co llection . The first follow-up data collection was conducted in 
Spring 1993. This data collection took about three and one-half hours, and included Form 88B 
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of the CAAP reading comprehension, mathematics, and critical thinking modules; the College 
Student Experiences Questionnaire (CSEQ) (Pace, 1984); and a follow-up instrument developed 
for the NSSL. The CSEQ and the NSSL follow-up instrument were used to measure a wide 
range of students’ experiences in the first year of college. 

Of the original sample of 3,840 students who participated in the Fall 1992 testing, 2685 
participated in the Spring 1993 data collection, for a first-year follow-up response rate of 
69.92%. To adjust for potential response bias by sex, race/ethnicity, and institution, however, a 
sample weighting algorithm was developed. Follow-up participants in each institution were 
weighted up to the institution’s first-year population by sex (male or female) and race/ethnicity 
(Caucasian, Black, Hispanic, other). Thus, for example, if institution A had 100 Black women in 
Its first-year class, and 25 Black women in the sample, each Black woman in the sample was 
given a sample weight of 4.00. An analogous weight was computed for participants in each sex 
X race/ethnicity cell in each institution. Applying sample weights in this manner allowed us to 
adjust not only for response bias by sex and race/ethnicity, but also for response bias (i.e., 
differential response rates) by institution. 

^ cond follow-up data collection . The second follow up of the NSSL sample was 
conducted m the Spring of 1994. Similar to the first follow-up, extensive measures of students’ 
second-year experiences were taken from their responses on the CSEQ and the NSSL follow-up 

survey. Students also completed Form A of the CAAP writing skills and science reasoning 
modules. 

Of the 2,685 students who participated in the first follow-up (Spring 1993), 1,761 
participated in the second follow-up (Spring 1994), for a response rate of 65.6%. A second 
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weighting algorithm was developed which was analogous to that applied in the first year to 
adjust for response bias by sex, race/ethnicity, and institution. 

Third follow- up da ta collection . The third follow-up of the NSSL sample took place in 
the Spnng of 1995 at the 18 four-year institutions. Measures of students' third-year experiences 
were taken from their responses on the CSEQ and the NSSL Follow-Up Survey. Participants 
also completed Form B of the CAAP reading comprehension and critical thinking modules. 

Of the 1,613 four-year college students who participated in the Spring 1994 data 
collection, 1,054 participated in Spring 1995, for a third-year response rate of 65.3%. A third 

weighting algorithm was developed to adjust for sample response bias by sex, race/ethnicity, and 
institution. 

Analytical Model 

The independent variable of primary interest in analyses of all three years of the study 
was the 10-item Cogmtive Level of Instruction Scale. The dependent variables/outcomes 
measures for each of the three years were the end-of-the-year scores on three CSEQ self-reported 
gains scales: wnting and thinking skills, understanding science, and understanding the arts and 
humanities. These scales were constructed from items on the CSEQ for which students indicated 
how much they felt they had gained or made progress in a variety of aspects of college learning 
and cognitive growth. 

Two sets of potentially confounding variables - individual-level variables and 
institutional-level variables - also were included in the analytic model. A number of factors 
extraneous to the study might influence students' involvement with peers, as well as their 
cognitive growth during college. As a consequence, simple correlations might yield a 
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misleading estimate of the impact of peer involvement on students' cognitive development (cf, 
Astin, 1993; Pascarella, 1985; Pascarella & Terenzini, 1991). 

In selecting individual-level confounding variables, we were guided by evidence on the 
factors independently influencing learning and cognitive development in college (cf , Astin, 

1968, 1977, 1993; Astin & Panos, 1969; Kuh, 1993; Pascarella, 1985; Pascarella & Terenzini, 
1991). Individual-level variables incorporated in the analytic model were: precollege (Fall 1992) 
cognitive development; race/ethnicity (white/non-white); sex; age; precollege (Fall 1992) 
academic motivation; socioeconomic status; total credit hours completed at the end of the first 
year; hours per week spent studying; on- or off-campus residence; hours employed per week; and 
the number of courses taken in social sciences, mathematics, technical/professional, arts and 
humanities, and natural science and engineering. 

Because evidence also suggests that the academic preparation of an institution’s student 
body can influence the climate of an institution (cf , Pascarella & Terenzini, 1991), an estimate 
of student academic preparation was considered an institutional-level confounding variable. The 
measure of student academic preparation was estimated with the average precollege (Fall 1992) 
composite cognitive development score (CAAP reading, math, and critical thinking) for the 
sample of first-year students at each of the 23 institutions. Each student in the sample was given 
the mean estimate of academic preparation for her or his institution, and the institutional mean 
estimate was used in the analysis of end-of-first-year cognitive development and the four areas of 
self-reported gams. Operational definitions of all variables in the analyses are shown in Table 1. 



Insert table 1 about here 



Analysis 



In the first stage of the data analysis we estimated the net impact of the Cognitive Level 
of Instruction Scale on the outcome measures while applying statistical controls for the. 
potentially confounding variables specified above. Using an ordinary least squares approach, 
each of the outcomes was regressed on all the potentially confounding variables and the 
Cognitive Level of Instruction Scale. Separate analyses were conducted for end-of-first-, 
second-, and third-year outcomes. 

The second stage of the analysis sought to determine if the potential cognitive effects of 
the Cognitive Level of Instruction Scale were general or conditional. That is, is the cognitive 
impact of these instructional processes similar in magnitude for all students in the sample 
(general effects), or does it differ in magnitude for different kinds of students (conditional 
effects)? To test for the presence of conditional effects, a set of cross-product terms was formed 
between the Cognitive Level of Instruction Scale on the one hand and the background variables 
on the other. The cross-product terms were then added to the general effects equations employed 
in the first stage of the analysis. A significant increase in explained variance (R^) due to the 
cross-product terms indicates the presence of significant conditional effects (Pedhazur, 1982). 

Useable data were available for 2,189 four-year college students in the analysis of end-of- 
first-year outcomes; 1,507 four-year college students in the analysis of end-of-second-year 
outcomes; and 1,017 four-year college students in the analysis of end-of-third-year outcomes. 

All analyses were conducted with weighted samples, corrected for the unweighted (actual) 
sample size to obtain correct standard errors. -Because of the relatively large unweighted sample 
sizes, the critical alpha level was set at .01. 
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Results 



End-of-First-Year Outcomes 

Table 2 shows the estimated net effects (expressed as Beta coefficients) of the Cognitive 
Level of Instruction Scale on the three end-of-first-year Gains scales. In the presence of 
statistical controls for the battery of confounding influences, the Cognitive Level of Instruction 
Scale had significant effects on all three of the outcome variables. Standardized and 
unstandardized regression coefficients can be found in Table 2. 



Insert table 2 about here 



For both the Gains in Writing and Thinking Analytically Scale and the Gains in 
Understanding Arts and Humanities Scale, the Cognitive Level of Instruction Scale had by far 
the greatest influence of any of the control variable, as indicated by the standardized coefficient 
(beta). In each of these cases, the beta associated with the Cognitive Level of Instruction Scale 
was more than double that of any other significant influences. When the Gains in Understanding 
Science and Technology Scale was the dependent variable, the Cognitive Level of Instruction 
Scale was still significant, but its beta was not the highest. 

As the table indicates, selected backgroimd and control variables also had significant 
effects on the dependent variables. The significant effects of gender and membership in a 
minority group influenced the decision to examine the sample for interactive effects (see below); 
the significance of the Academic Motivation Scale will be described in the Discussion section of 
the paper. 
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End-of-Second-Year Outcomes 



Second year results are very similar to those of the first year. Again, in the presence of 
background and control variables, the Cognitive Level of Instruction Scale had significant effects 
on all three of the outcome variables. Standardized and uhstandardized regression coefficients 
can be found in Table 3. 



Insert table 3 about here 



As in the first year, the Cognitive Level of Instruction Scale had by far the greatest 
influence of any of the control variables on both the Gains in Writing and Thinking Analytically 
Scale and the Gains in Understanding Arts and Humanities Scale, as indicated by the 
standardized coefficient (beta). And when the Gains in Understanding Science and Technology 
Scale was the dependent variable, the Cognitive Level of Instruction Scale was still significant, 
although its beta was not the highest. 

In the second year, the influence of number of science courses taken on Gains in 
Understanding Science and Technology, and the influence of number of arts and humanities 
courses taken on Gains in Understanding Arts and Humanities are also worth noting. 
End-of-Third-Year Outcomes 

Third year results exhibit the same pattern as those of the first two years. Again, in the 
presence of background and control variables, the Cognitive Level of Instruction Scale had 
significant effects on all three of the outcome variables. Standardized and imstandardized 
regression coefficients can be found in Table 4. 
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Insert table 4 about here 



Additional Analysis 

Conditional Effects 

Because we found that gender to be significant in the full model for Gains in 
Understanding Science and Technology in years 1 and 2 , and membership in a minority group to 
be significant for Gains in Understanding Science and Technology in year 1 and Gains in 
Understanding Arts and Humanities in years 2 and 3, we decided to test these outcomes for 
conditional effects. Hence we created a set of cross-product terms between the Cognitive Level 
of Instruction Scale and the other variables in the model. We then added these terms to the full 
model to test for interactions. 

The addition of the sets of cross-product terms was associated with modest, but 

statistically significant r 2 increases (less than 2%) in each case. However, none of the variables 
of interest were significant in this regression, that is, there were no interactive effects. Based on 
this analysis, there is no reason to believe that the effects of the Cognitive Level of Instruction 
Scale on the dependent variables differ by gender or by ethnic group. For example, although 
being a male student is associated with significant Gains in Understanding Science and 
Technology, the impact of the Cognitive Level of Instruction Scale on that outcome is not 
significantly different for male and female students. 



Summary and Discussion 



Previous research has indicated that classroom instructional processes delivered at a 
reasonably high intellectual level have positive associations with student gains in general 
cognitive skills. The present longitudinal study of students in 18 four-year institutions examined 
the extent to which higher-order cognitive instruction, as defined in Bloom’s (1956) taxonomy, 
influenced students’ cognitive development during each of the first three years of college. 
Specifically, we investigated the net effects of these instructional practices, the extent to which 
the effects varied over time, and the extent to which the effects were general or conditional for 
this student population. With controls made for an extensive set of confoimding influences, 
including precollege cognitive development, the Cognitive Level of Instruction Scale had 
significant, and overwhelmingly positive, effects on self-reported measures of student self- 
reported gains in general cognitive skills in each of the three years of the study. Of special note 
is the finding that the variable representing instructional practices that stressed higher-order 
thinking exerted the strongest influence on student gains in writing and analytical thinking in 
each of the three years. Additional analyses found no conditional effects for the cognitive level 
of instruction scale, thus indicating that the influence of this scale did not differ by gender or 
ethnicity for this population. 

One interesting finding deals with the significance of an “Academic Motivation” Scale on 
the self-reported gains scales. This variable was significant in each of the three years and for 
each of the three outcomes, often with an effect nearly as strong as that of the Cognitive Level of 
Instruction Scale, as measured by the standardized regression coefficient (beta). We hypothesize 
that students who are highly motivated to succeed may be more aware of what transpires in the 
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classroom, and of variations in instruction; they also may be better at monitoring their progress 
and assessing the impact of teaching methods on their learning. 

When we talk about the ability to think critically about issues, we are generally referring 
to the skills and capabilities individuals possess that allow them to perform the following tasks: 
identify central issues and assumptions in an argument; recognize important relationships; make 
correct inferences from data; draw conclusions from information or data; interpret whether 
conclusions are warranted based on give data; and evaluate evidence or authority (Furedy and 
Furedy, 1985). In the presence of teaching practices designed to elicit higher-order thinking, 
students in this study consistently reported gains in their ability to perform these intellectual 
tasks, both in general, and in specific discipline areas (i.e., Art/History/Humanities and Science 
and Technology). These findings add to our understanding of the factors which foster students’ 
acquisition of these general Cognitive skills, and thus have implications for practitioners who 
seek to assess and improve the quality of American undergraduate education. While much of the 
interest in assessment has focused on the desired cognitive outcomes of higher education, little 
attention has been paid to the instructional processes that influence these outcomes (Braxton and 
Nordvall, 1996). With this study, we have gained insight into specific cognitive aspects of 
classroom instruction that can influence students’ intellectual growth. A dominant myth 
surrounding the acquisition of higher-level thinking is that these complex skills are a natural by- 
product of knowledge acquisition. However, as Bloom (1956) has noted, the opposite is actually 
true - such skills do not develop automatically, but require deliberate instruction. The current 
research supports the notion that specific types of instruction can impact students’ growth; it 
further suggests that the college classroom and the educational encounters that occur within can 
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not be discounted in the on-going effort to better understand how students change and grow 
during college. 
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Table 1— Variables 



Background Variables 

1 . Individual precollege (Fall 1992) scores on either the critical thinking module of the CAAP critical 

thinking score, or on the composite measure combining math, reading comprehension, and 
critical thinking CAAP scores. 

2. Precollege (Fall, 1992) academic motivation: an 8-item, Likert-type scale (5 = strongly agree to 1 = 

strongly disagree! with an internal consistency reliability of .65. The scale items were developed 
specifically for the NSSL and were based on existing research about academic motivation (e.g.. 
Ball, 1 977). Among the constituent items were the following: "I am willing to work hard in a 
course to learn the material, even if it won't lead to a higher grade," "When I do well on a test it is 
usually because I was well prepared, not because the test was easy," "In high school I frequently 
did more reading in a class than was required simply because it interested me," and "In high 
school I frequently talked to my teachers outside of class about ideas presented during class." 

3. Gender: coded: 1 = female, 0 = male. 

4. Ethnicity: coded: 1 = non-white, 0 = white. 

5. Age 

6. Socio-economic Status: a three item scale comprised of standardized values for “Mother’s Education,” 

“Father’s Education,” and “Family Income” (taken from the pre-college survey). 

7. Residence: coded: 1 = lived off campus; 2 = lived on campus 

8. Number of credit hours taken: total number of credit hours each student expected to complete during 

the first year of college (taken from the follow-up questionnaire). 

9. Number of hours worked: total number of hours a student worked per week both on- and off-campus 

(taken from the follow-up questionnaire). 

10-14 Number of courses taken in five areas. Respondents were given 61 different courses across the five 
broad areas to select from, and they were asked to indicate how many of each of the 61 courses' 
they had taken during the year. In the first year, the number of freshman courses was used; in the 
third year, each variable reflects the sum of freshman, sophomore, and junior year courses. This 
information was taken from the follow-up questiormaires. 

10. Natural sciences (Astronomy, Biology, Botany, Chemistry, Engineering, Geology, Microbiology, 

Physics, Zoology) 

11. Arts and humanities (Art history. Art Appreciation, Studio Art, Dance, Theater, Music Appreciation, 

Music Performance, Composition or Writing, English Literature, Foreign Language, Humanities, 
Women’s Studies, Latin-American Studies, Afncan-American Studies, Philosophy, Linguistics, 
Classics, Religious Studies ) 
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Table 1-- Variables (cont.) 



12. Social sciences (Anthropology, Audiology/Speech Pathology, Child and Family Studies, 

Communications, Economics, Geography, History, Political Science, Psychology, Sociology, 
Social Work) 

13. Mathematics (Pre-Algebra, Algebra, calculus. Statistics, Computer Science, Geometry, Matrix 

Algebra, Accounting, Business Math) 

14. Technical or preprofessional (Drawing, Drafting, Architectural Design, Criminology, Education, 
Study Skills, Reading Skills, Agriculture, Business, Physical Therapy, Pharmacy, Physical 
Education, Nursing, Computer Programming). 

15. Average precollege (Fall 1992) CAAP composite scores for the sample of incoming students at each 

of the 1 8 institutions. 

16. Number of Hours spent Studying-(total number of hours each week that the student spend studying 

(taken fi-om the follow-up questionnaire). 

Independent Variable 

17. Cognitive Processes— a 10-item Likert-type scale (1 = never to 4 = very often) with an internal 

consistency reliability of .85. Scale items are fi-om the follow-up survey. Questions included; 
“Instructors engage me in classroom discussion or debate of course ideas and concepts,” “ 
Instructors ask challenging questions in class,” “ Instructors’ questions in class ask me to show 
how a particular course concept could be applied to an actual problem or situation,” “Instructors’ 
questions in class ask to argue for or against a particular point of view,” “Course exams require 
me to write essays and/or solve problems,” “Course exams require me to compare and contrast 
dimensions of course content,” “Course exams require me to point out the strengths and 
weaknesses of a particular argument or point of view,” “Examinations require me to argue for or 
against a particular point of view and defend my argument,” “Course papers or research projects 
require me to apply course content to solve an to actual problem,” “Course papers or research 
projects require me to argue for or against a particular point of view and defend my argument.” 

Dependent Variables 

18-20. Gains Scales in three areas. Scale items are from the CSEQ. Students were asked; “In thinking 
over your experiences in college up to now, to what extent do you feel you have gained or made 
progress in each of the following respects?” 

18. Gains in Understanding Science and Technology— a 4-item Likert-type scale (1 = very little to 4 = 

very much) with an internal consistency reliability of .86. Specific items included; 
“Understanding the nature of science and experimentation,” “Understanding hew scientific arid 
technical developments,” “Becoming aware of the consequences (benefits, hazards, dangers, 
values) of new applications in science and technology,” “Quantitative thinking— understanding 
probabilities, proportions, etc.” 

19. Gains in Broadening Thinking About Art/History/Humanities — a 5-item Likert-type scale (1 = very 
little to 4 = very muchl with an internal consistency reliability of .76. Specific items included; 
“Developing an understanding and enjoyment of art, music, and drama,” “Broadening your 
acquaintance and enjoyment of literature,” “Becoming aware of different philosophies, cultures, and 
ways of life,” “Seeing the importance of history for understanding the present as well as the past,” 



Table 1-- Variables (cont.) 



acquaintance and enjoyment of literature,” “Becoming aware of different philosophies, cultures, and 
ways of life,” “Seeing the importance of history for understanding the present as well as the past,” 
“Gaining knowledge about other parts of the world and other people- Asia, Africa, South America, 



20. Gains in Writing and Thinking Analytically--a 4-item Likert- type scale (1 = very little to 4 = very 
muchl with an internal consistency reliability of .77. Specific items included: “Writing clearly 
and effectiyely,” “Ability to think analytically and logically,” “Ability to put ideas together, to 
see relationships, similarities, and differences between ideas,” “Ability to learn on your own, 
pursue ideas, and find information you need.” 
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TABLE 2 



Regression Analysis Summaries— First Year 



PREDICTOR 


Perceived Gains in 
Understanding 
Science 


Perceived Gains in 
Analytical Skills 


Perceived Gains in 
Understanding 
Arts 


Individual Composite Cognitive 


-009 


.002 


-.014 


Development Scores 


(-.028) 


(.008) 


(-.054) 


Average Precollege Composite 


.023 


-.018 


.036* 


Cognitive Development 


(.044) 


(-.043) 


(.079) 


Precollege Academic Motivation 


.274* 


.114* 


.144* 




(.191) 


(.097) 


(.117) 


Gender 


-.267 


.035 


.021 




(-.175) 


(.028) 


(.016) 


Minority 


.097* 


-.020 


.019 




(-.063) 


(-.016) 


(.014) 


Age 


-.013* 


-.006 


-.027 




(-.084) 


(-.051) 


(-.020) 


Number of Credit Hours Taken 


.011 


.013 


.0009 




(.020) 


(.030) 


(.002) 


Number of Hours Worked 


.0009 


-.015* 


.005 




(.003) 


(-.067) 


(.024) 


Number of Courses Taken in the 


.183* 


.134 


.121* 


Natural Sciences or Engineering 


(.092) 


(.083) 


(.071) 


Number of Courses Taken in the 


-.136* 


-.018 


.087 


Arts/Humanities 


(-074) 


(-.012) 


(.056) 


Number of Courses Taken in the 


-.046 


-.012 


.035 


Social Sciences 


(-.025) 


(-.008) 


(.0422 


Number of Courses Taken in 


-.122 


-.005 


-.007* 


Mathematics 


(-.077) 


(-.004) 


(-.057) 


Number of Courses Taken in 


-.030 


-.003 


.004 


Technical/Professional 


(-.048) 


(-.006) 


(.009) 




-.002 


.036 


.053 


Campus Residence 


(-.002) 


(.029) 


(.041) 




.004 


-.001 


-.003 


Socio-economic Status 


(.013) 


(-.005) 


(.-.012) 




.053* 


.038* 


.017 


Hours Spent Studying 


(.092) 


(.082) 


(.034) 




.228* 


.415* 


.358* 


Cognitive Level of Instruction 
Scale 


(.156) 


(.349) 


(.286) 


R^ 


.151 


.186 


.142 



Note: Top number is the metric or unstandardized coefficient; number in parentheses is the standardized (beta) 
coefficient . * p < .01 
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TABLE 3 



Regression Analysis Summaries-Second Year 



PREDICTOR 


Perceived Gains in 
Understanding 
Science 


Perceived Gains in 
Analytical Skills 


Perceived Gains in 
Understanding Arts ' 


Individual Composite Cognitive 


-.017 


-.007 


-.024* 


Development Scores 


(-.050) 


(-.027) 


(-.089) 


Average Precollege Composite 


.015 


-.019 


.029 


Cognitive Development 


(.028) 


(-.046) 


(.069) 


Precollege Academic 


.267* 


.178* 


.144* 


Motivation 


(.171) 


(.151) 


(.116) 


Gender 


-.174* 


.044 


.029 




(-.106) 


(.036) 


(.022) 


Minority 


-.022 


-.008 


.094* 




(-.013) 


(-.007) 


(.070) 


Age 


-.014* 


-.013* 


-.003 




(-.086) 


(-.108) 


(-.025) 


Number of Credit Hours Taken 


-.007 


.008 


-.028* 




(-.013) 


(.019) 


(-.062) 


Number of Hours Worked 


.0009 


.007 


.013 




(.030) 


(.032) 


(.057) 


Number of Courses Taken in the 


.107 


.009 


-.006 


Natural Sciences or Engineering 


(.443) 


(.052) 


(-.033) 


Number of Courses Taken in the 


-.015 


.014* 


.029* 


Arts/Humanities 


(-.088) 


(.110) 


(.210) 


Number of Courses Taken in the 


-.0009 


.008 


.008 


Social Sciences 


(-.042) 


(.052) 


(.045) 


Number of Courses Taken in 


.015 


-.003 


-.028* 


Mathematics 


(.042) 


(-.012) 


(-.103) 


Number of Courses Taken in 


. -.002 


-.009 


-.011 


Technical/Professional 


(-.007) 


(-.034) 


(-.038) 


Campus Residence 


-.038 


.044 


.242* 




(-.023) 


(.036) 


(.186) 


Socio-economic Status 


.012 


.0009 


.0002 




(.035) 


(.004) 


(.001) 


Hours Spent Studying 


.052* 


.055* 


-.011 




(.087) 


(.122) 


(-.023) 


Cognitive Level of Instruction 


.274* 


.401* 


.385* 


Scale 


(.171) 


(.332) 


(.302) 


2 

R 


.380 


.240 


.288 



Note: Top number is the metric or unstandardized coefficient; number in parentheses is the standardized (beta) 
coefficient * p < .01 
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Individual Composite Cognitive 
Development Scores 

Average Precollege Composite 
Cogmtive Development 

Precollege Academic Motivation 

Gender 

Minority 

Age 

Number of Credit Hours Taken 

Number of Hours Worked 

Number of Courses Taken in the 
Natural Sciences or Engineering 

Number of Courses Taken in the 
Arts/Humanities 

Number of Courses Taken in the 
Social Sciences 

Number of Courses Taken in 
Mathematics 

Number of Courses Taken in 
Technical/Professional 

Campus Residence 
Socio-economic Status 
Hours Spent Studying 

Cognitive Level of Instruction Scale 

2 

R 



.031 

(.054) 

.345* 

(.213) 

-.018 

(-. 011 ) 

-.119 

(-.069) 

-.017 

(-.104) 

.029 

(.048) 

.012 

(.039) 

.060* 

(.415) 

-.016* 

(-.119) 

-.011 
. (-.069) 

.012 

(.054) 

.002 

(.007) 

.026 

(.016) 

-.022 

(-.061) 

.030 

(.048) 

.286* 

(.174) 

.309 



roefficiInT*°p '^“^‘“d^dized coefficient; number 



.033* 


-.011 


(.118) 


(-.039) 


-.005 


.035 


(-.011) 


(.079) 


.161* 


.228* 


(.132) 


(.182) 


.074 


-.024 


(.058) 


(-.018) 


.084 


.116* 


(.065) 


(.087) 


-.005 


.002 


(-.046) 


(.013) 


.083 


-.011 


(.177) 


(-.022) 


.007 


.012 


(.031) 


(.049) 


-.004 


-.003 


(-.003) 


(-.023) 


.004 


.026* 


(.046) 


(.266) 


-.002 


.008 


(-.014) 


(.071) 


.006 


-.014* 


(.038) 


(-.079) 


-.004 


-.008 


(-.025) 


(-.045) 


.114* 


.053 


(.091) 


(.041) 


-.007 


.005 


(-.027_ 


(.018) 


.083* 


.026 


(.177) 


(.054) 


.461* 


.382* 


(.370) 


(.299) 


.285 


.402 



in parentheses is the standardized (beta) 
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